Testing Techniques for Ada 95

ions that are designed in this way, using a private child package to declare package body data, are not only easier to test but are also more easily extended. Besides their usefulness as a replacement for test points, hierarchical library units can also improve the testability of a system in other ways. In particular, there are many operations which are specific to testing that would benefit from increased visibility of an abstraction. A good candidate for this type of operation is a “check” procedure which can be used during testing to verify that an object has a particular value. Increased visibility of the internals of an abstraction allows this procedure to take advantage of the full view of a type to give improved diagnostics when the check fails. The great benefit of using child units to implement these utilities is that they can be made available during testing and removed when no longer needed, without affecting any other packages. The example below illustrates the use of a child package to provide testing-specific operations for the Complex type. The Is_Valid operation can be used to ascertain that a Complex object satisfies its invariants, and may be implemented using the new Ada 95 Valid attribute. Careful consideration must be given before adding testing utilities to a package because they introduce an unwanted dependency between the test and the implementation details of the abstraction. This becomes a problem if the utilities are added on a per-test basis because any change to the representation of the abstraction requires them all to be updated. To avoid this, the utilities should be considered part of the design of the abstraction, and a few general purpose operations should be added rather than many test specific ones. As a general rule the dependencies of a test should be a subset of the dependencies of the software under test. This ensures that the test will only need modification when the software under test changes. SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 8 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com 2.1.2 The Impact of the Hierarchical Library Upon Testing We have shown that hierarchical library units are a valuable tool when implementing unit and integration level tests, because they allow us an unrestricted view of the private part of a package. Private child units also provide us with a simple idiom for structuring the body of a package so that declarations that are needed during testing are readily available. In this section we consider the testing issues that arise when hierarchical library units are used, not for testing per se, but to improve the properties of a design. As mentioned previously public child units can be used to partition different aspects of an abstraction, or to extend an existing abstraction with new functionality. We expect the latter to become common in Ada 95 due to the increased use of object-oriented techniques and the emphasis that they place upon programming by extension. In both of these cases the techniques that we have described in the previous section are sufficient to gain access to all of the information that is needed during testing. The example below illustrates the typical use of a child package to segregate operations that are only of interest to a limited number of clients: There are several ways in which this pair of packages can be tested. The first alternative is to write a combined test for both sets of operations, using a child package of Message.Tracking to gain visibility of both private parts. The obvious problem with this approach is that we lose the independence between the core abstraction and the tracking extensions; although we can use the message abstraction on its own we cannot test it without the tracking extensions. A better approach is to implement a stand-alone test for the core abstraction and then to test the tracking operations separately. During testing it is important to remember that the operations of a child package have privileged access to the internals of their parent package. As a result it is possible for a child operation to break some of the existing operations of the parent package which were considered adequately tested. This means that an adequate test for the child package, as identified by Harrold [10], must re-test all of the operations of the core abstraction which may have broken. Without careful analysis it is difficult to tell which operations will need to be re-tested in the light of the new package. SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 9 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com Private child units will often be used to structure the implementation of a complex abstraction as a sub-system of packages. For example the Message abstraction shown above may be implemented using a queue of incoming and outgoing messages. The fact that the abstraction is implemented using a queue is of no significance to its clients, and so a private child package is used to hide the details. The private descendants used in implementing a sub-system should be subject to the same tests as other units in the system, and in practice private child units introduce no extra difficulty for the test engineer. The queue package from the Message sub-system is shown below, together with a child subprogram used to test it: Note that this child subprogram can be made the main subprogram of the test even though it is a private descendant of the Message package. There is at least one other motivation for the implementation of a design using a set of hierarchical library units and that is to avoid polluting the library name-space. The structure of the Ada 95 predefined library hierarchy is in part motivated by this. The use of hierarchical library units for this purpose does not introduce any new testing considerations. 2.1.3 Recommendations  Use a public child unit instead of a test point to gain visibility of the internals of an abstraction.  Declare data and types that are local to the implementation of an abstraction in a private child instead of the package body. This makes the declarations available during testing, and also to extensions.  At design time use a child package to implement utilities that will be useful during testing, e.g. check functions.  Perform an integration test of the operations of a child unit with those of its parent. This ensures that the child unit has not abused its privileged visibility and detrimentally affected the operations of its parent.  Avoid extending an abstraction using a child package when the visibility of a normal client will suffice. This will avoid unnecessary re-testing of the parent abstraction. SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 10 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com 2.2 Protected Objects Protected objects were added to Ada 95 to provide a passive and data oriented means of synchronising access to shared data. These objects can be considered similar to packages or tasks in that they are divided into a distinct specification and body. The specification of the protected object encapsulates the data that is being shared, and defines the interface used to access the data. The body of the protected object provides its implementation. The interface to a protected object is defined by its protected operations, either procedures, functions or entries, which the language guarantees will have exclusive access to the object’s data. The example below uses the familiar concept of a quantity, or counting semaphore to illustrate the typical use of protected objects. For a detailed discussion of the implementation of semaphores and other building blocks using protected objects see [1]. Protected procedures and entries have read and write access to the shared data and consequently require exclusive access to the object. Protected functions have read-only access and therefore more than one function can be active simultaneously. 2.2.1 Testing Protected Objects The Ada 95 Style Guide [6] recommends that the time spent inside protected operations is kept short so that the protected object is locked for as little time as possible. If this guideline is followed, the amount of code involved in the implementation of the protected object will be small. As a result the overhead of isolation testing each protected operation is unacceptably high, and we therefore recommend that the protected object as a whole is considered for isolation testing. Further motivation for testing at this level is the difficulty of isolating protected operations from each other without changing the protected object. The usual means of obtaining this independence is to use separate compilation, but for reasons of efficiency Ada 95 does not allow protected operations to be declared separate. A problem encountered when testing protected objects, as with any software that encapsulates state, is that there will often be areas of functionality that are difficult to exercise in a test environment. In order to make this functionality testable it is useful to have access to the shared data inside the protected object so that we can manipulate it directly. This can be realised by adding operations to the object’s interface which access each protected data item. This allows the protected object to be conveniently set into any state that is required during testing. It is interesting to note that neither of the techniques described in section 2.1 can be used to improve the testability of protected objects: the child unit approach is ruled out because protected objects cannot have children; and the test point technique is not practical because of the lack of separate compilation. SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 11 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com The example below shows the semaphore example modified for testing: The problem with adding testing operations to the protected object is that there is nothing to distinguish them from the object’s normal interface. This distinction is important because the testing interface must not be used by any normal clients of the object. One solution to this problem is to encapsulate the protected object inside the private part of a package. This allows the normal interface to be exported from the package while the testing interface remains hidden inside the private part The example below shows the semaphore example modified to add a hidden testing interface: To utilise the testing interface all tests are written inside a child unit of the Semaphore package, from where they have visibility to both the normal and testing interfaces: SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 12 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com The disadvantage of this structure is that all children of the Semaphore package have access to the testing operations of the protected object. An alternative encapsulation strategy is to declare the protected object inside the body of a package and use the test point technique [4] to give test code visibility to the testing operations. Testing is performed by replacing the normally null implementation of the test point with test code which can then freely utilise the testing operations. It is important to note that there are circumstances in which a protected entry must be called directly, not via an interface subprogram. A good example of this is a protected entry which is the target of a timed or conditional entry call, or the trigger of an asynchronous transfer of control. 2.2.2 Testing Clients of Protected Objects Protected objects can have a negative impact on the testability of parts of a system that use them unless care is taken to ensure that they can be sufficiently de-coupled from the rest of the system. When the client of a protected object is subject to isolation testing all calls to the protected operations must be simulated. This would normally be achieved by writing “stubs” for the called operations using a suitable test harness [11]. The efficient use of this technique relies upon the simulated operations being declared separate. As mentioned above, Ada 95 does not allow protected operations to be declared separate, so during testing the protected body as a whole must be simulated. An attractive alternative is available if the protected object is encapsulated inside a package as described above. In this situation it is the interface subprograms of the package which, when declared separate, are simulated, rather than the protected operations themselves. SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 13 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com 2.2.3 Recommendations  Consider testing protected objects as a whole rather than isolation testing each operation.  Define testing operations for each protected object, to allow direct manipulation of the object’s state.  If testing operations are added, shield them from the user by defining a testing interface.  Consider using a testing interface to de-couple the protected object from its clients during isolation testing.  Alternatively, declare the body of the protected object separate to allow it to be simulated. 2.3 Controlled Types Regardless of how well an abstraction is designed and implemented, if it is not used correctly then it will not work. Probably the most common misuses of an abstraction are incorrect initialisation and inadequate clean up when it object goes out of scope. The Ada language exacerbates this problem by allowing a scope to be left in many different ways, for example by the propagation of an exception, an abort, a return statement, or an asynchronous transfer of control. In Ada 95 the designer of an abstraction can ensure that these operations happen correctly under all circumstances by using Controlled Types. A controlled type is created by derivation from one of the predefined types in package Ada.Finalization, and from this type it inherits either two or three of the following subprograms:  Initialize, which is called to provide default initialisation of a new object;  Finalize, which is called to clean up an object when it is no longer needed;  Adjust, which is called during assignment to modify Ada’s default bit-wise copy. Note that adjustment is only available for non-limited types. The default implementation of these subprograms can be overridden by the derived type, and is automatically called by the Ada runtime system when an object of the type is created, deleted or assigned. The example below, which is taken from the Ada 95 Rationale [3], illustrates the use of a controlled type for the safe management of a resource: SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 14 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com In this example the creation of a Handle object causes the Initialize procedure to be called, which in turn locks the specified resource. When the object goes out of scope, no matter how the scope is left, the Finalize procedure guarantees that the resource is unlocked. 2.3.1 The Impact of Controlled Types Upon Testing It is expected that many abstractions will take advantage of the benefits of controlled types to provide a cleaner, more reliable interface. When a controlled type is used for this purpose it is considered an implementation detail of the abstraction and its use is hidden from the abstraction’s clients. This is of course advantageous because the abstraction can then be re-written to add or remove the controlled type without changing the remainder of the system. It is unfortunate then that the use of a controlled type often emerges during testing and must be considered part of the abstraction’s interface. To understand why this happens consider the way in which the simple abstraction shown below, which is built using the Handle abstraction, might be tested : SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 15 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com Assuming that the isolation testing strategy is used to test this abstraction, all calls to subprograms that are not part of the software under test must be simulated. This causes a problem because the controlled operations are implicitly called during the test whenever a Resource_Wrapper object is created or deleted. Like the other operations the controlled operations can be simulated, but doing so introduces a dependency between the test and the implementation of the abstraction. For example the test for Resource_Wrapper must change if the Handle abstraction is rewritten to avoid the use of controlled types. In practice the problem is made worse if the software under test is implemented using several different abstractions, all of which use controlled types. In this situation the test transitively depends upon the implementation of all of the abstractions. The simulation of controlled operations and the dependency that this introduces can be avoided by using the operation’s real implementation during testing. However, the integrity of the test can only be guaranteed if the controlled operations have already been thoroughly tested. A large part of the system may be involved in the verification of the controlled types, particularly when they have a non-trivial implementation, and many of the benefits of isolation testing are lost. A solution to this problem is to provide a null implementation of the controlled operations during testing. This enables the test to remain independent of the controlled nature of the abstraction because the offending operations need not be simulated. In addition the implementation of each null operation is trivial and therefore does not need to be tested before it is used. It is important to note that this strategy will only work if the encapsulation provided by an abstraction is complete, in other words the software under test must not directly depend upon the actions of the controlled operations. The elided example below helps illustrate this point: A controlled abstraction, like any other abstraction, must at some point be tested. During testing the controlled operations are considered part of the software under test and are not simulated. However, the subprograms that are called by the controlled operations are simulated and this may cause difficulties. The number of calls made to controlled operations will often be large; every time a controlled object is created, deleted or assigned a call is made. This places a significant burden upon the tester who must specify the order of these calls. In some situations it may also be difficult to identify the precise order in which the calls occur. This is due to the implementation permissions granted by the ARM for non-limited controlled types. A good example of the use of such permission is in the elimination of temporary objects during the assignment of controlled types. In practice these problems only occur when a controlled operation results in a call to a subprogram that is outside the software under test. Fortunately most of the common uses of controlled types will not cause this problem. SOFTWARE QUALITY ASSURANCE | TOOLS & TECHNOLOGY | PROFESSIONAL SERVICES | ACADEMY P a g e | 16 Testing Techniques for Ada 95 © Copyright QA Systems GmbH 2016 www.qa-systems.com 2.3.2 Recommendations  Do not simulate calls to controlled operations during testing; simulating these calls introduces a dependency between the test and the use of the controlled type.  During testing, provide a null implementation of any controlled operations that are not part of the software under test. These operations can then safely be called as part of the test.  Be aware that difficulties may arise when testing a controlled abstraction. In particular remember that the implementation permissions granted by the ARM may result in platform specific tests. 3 Summary and Conclusions In this paper we have presented several techniques that can be used to improve the testability of software written in Ada 95. In particular we have focused our attention upon three features of the Ada language: the hierarchical library, protected objects, and controlled types. We have shown that the hierarchical library has the greatest impact upon the testing process because it provides a convenient method of accessing the information hidden inside an abstraction. The lack of visibility of this information was a significant obstacle to achieving testability when using Ada 83. We have also shown that the full benefits of the hierarchical library can only be realised if testing is considered at an early stage in the development process. We followed our discussion of the hierarchical library by considering some of the difficulties encountered when testing software that uses protected objects; the effects of controlled types were also considered. Again, the techniques that we recommend emphasise that early consideration of the testing process is essential if testing is to be both thorough and efficient.